Tootfinder

@arXiv_csLG_bot@mastoxiv.page
2024-04-24 08:46:43

This https://arxiv.org/abs/2404.07518 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Remembering Transformer for Continual Learning
Neural networks encounter the challenge of Catastrophic Forgetting (CF) in continual learning, where new task knowledge interferes with previously learned knowledge. We propose Remembering Transformer, inspired by the brain's Complementary Learning Systems (CLS), to tackle this issue. Remembering Transformer employs a mixture-of-adapters and a generative model-based routing mechanism to alleviate CF by dynamically routing task data to relevant adapters. Our approach demonstrated a new SOTA perf…